Require unique scratch filenames and define the structurally-red-anchor path - #121
Merged
dmccoystephenson merged 2 commits intoAug 16, 2026
Merged
Conversation
…or path Three defects observed empirically in a 10-way parallel fan-out on 2026-08-15: - A generically-named scratchpad file holding one repo's PR body was silently overwritten by a concurrent dev loop. Phase 3 now requires scratch filenames unique per repo and per cycle, Phase 4 composes the PR body by file, and the concurrent-session edge case names the scratchpad as a third shared resource. - A repository whose required jobs have never succeeded on the base branch had no defined disposition, so every cycle terminated in an ordinary hand-off. Phase 1 now establishes whether a merge path exists, Phase 4 permits a per-job signal assessment, and Phase 8 names the blocking condition. - A build tool reporting no test sources still exits successfully. Phase 3 now requires confirming the anchor executed tests before recording a PASS. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
8 tasks
6 tasks
dmccoystephenson
added a commit
that referenced
this pull request
Aug 29, 2026
The Docs rubric item was an anchor-shaped score with no anchor behind it, because the check it named ran three phases later. That is the same defect as finding 1's PR #121 entry, relocated from the signal to the schedule. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Aug 29, 2026
dmccoystephenson
added a commit
that referenced
this pull request
Aug 29, 2026
…138) * Ground Phase 4's Docs score in a real pass and fix two Phase 3 rules Phase 4's Docs rubric item named a check that Phase 7 does not perform until three phases later, so the item could only ever be scored from judgment — the thing the rubric's own framing forbids — and a clean Phase 7 sweep left no trace that would distinguish it from a skipped one. The table pass now happens at Phase 4, where the score is taken; Phase 7 becomes a re-check scoped to whatever Phases 5-6 changed, with an explicit instruction to record that it ran. Phase 3's zero-tests rule ended by describing {{TEST_CMD}} as a substitution, which is the one context a generated child skill never has: after substitution it read "fall back to whatever the ./gradlew test substitution names". It now names the executed-test count as the gate, which reads correctly on both sides of substitution. A scan of the rest of the template body found no other sentence referring to a token as a token. Phase 3's scratch-file rule gains a third cleanup form observed to work where a bare rm is blocked: git clean -f on a single untracked path inside the working tree, previewed with git clean -n. Closes #126 Closes #134 Closes #137 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Record PR #138 under RESEARCH.md finding 1 The Docs rubric item was an anchor-shaped score with no anchor behind it, because the check it named ran three phases later. That is the same defect as finding 1's PR #121 entry, relocated from the signal to the schedule. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three defects were observed empirically during a 10-way parallel dev-loop fan-out on 2026-08-15. All three are properties of the base template rather than of any one generated skill, so they are fixed here.
1. Shared scratchpad filenames collide between concurrent cycles — motivated by dmccoystephenson/simpleskills-dev-loop#9. A food-spoilage cycle silently overwrote the scratchpad file holding a SimpleSkills PR body mid-cycle. Nothing was mis-posted only because
gh pr create --body-filehad already run; a slightly different ordering (write body, wait on CI, then use the file) would have published one repository's PR description onto another repository's PR.<repo>-<branch>-pr-body.md, never a genericpr-body.md), requiring the file to be written immediately before the command that consumes it, and requiring re-verification viagh pr view <number> --json bodywhere a body has to survive a wait.gh pr createfence is switched from--body "..."to--body-file, which is what the neighbouring command-substitution rule already prescribes for every other long body..self-review-scratch.md.origin/<default-branch>, now names the scratchpad as a third contended resource and notes that unlike the first two it fails silently.2. No disposition for a repository whose CI can never be green — motivated by dmccoystephenson/beyond-nations-dev-loop#1. On
Preponderous-Software/beyond-nationsthe Unity jobs have failed on every recorded run since February 2026 because the license secrets are empty, so the green-anchor merge gate can never be satisfied and every cycle terminates in a hand-off regardless of quality — including documentation-only changes for which those jobs carry no signal either way.3. A vacuously green build anchor — motivated by dmccoystephenson/food-spoilage-dev-loop#3.
./gradlew testprintsBUILD SUCCESSFULon:test NO-SOURCE, and the pytest/npm equivalents exit 0 on "no tests ran".The second half of that issue (exporting
JAVA_HOMEbefore invoking./gradlew) is a repo-specific finding and was deliberately not back-ported here, per CLAUDE.md's "What belongs here vs. in generated skills".No new placeholders are introduced and no phase numbers change, so no substitution-table or README changes are required.
Research grounding
Defects 2 and 3 both implement RESEARCH.md §1 (self-critique without an external signal is unreliable). A required job that has never passed, and a green that executed zero tests, are both anchors that carry no information; treating either as the external signal degrades the self-review to exactly the unanchored critique that finding shows is neutral-to-harmful. The existing "Green CI is not verification when CI's scope excludes the changed files" rule from PR #54 is the same argument applied to a different way an anchor can be uninformative, and these two changes are written as its neighbours. An Implementations entry is added under §1.
Defect 1 (scratch-file collisions) is a harness-concurrency hazard, not a model-behavior question: no finding in RESEARCH.md applies, and none is claimed.
Doc sync check
README.md's "What it does" Step list still matchescreate-dev-loop.md's Steps 1:1 — no Steps were added, removed, or renamed{{placeholder}}added or changed has a corresponding Step 4 substitution-table row — no new placeholders; the added text reuses{{EXTERNAL_SIGNAL_LABEL}},{{DEFAULT_BRANCH}}, and{{TEST_CMD}}RESEARCH.mdupdated (an Implementations entry under §1 for the anchor-integrity changes)Test plan
python3 scripts/check_docs.py— passes (0 errors).python3 -m pytest tests/test_check_docs.py -q— 13 passed./create-dev-loopagainst a real repository end-to-end) was not run for this change, and is marked UNVERIFIED. The change is additive prose inside the template block: fences remain escaped, phase numbers and the Step list are untouched, and no placeholder was added, so the mechanical checks cover the failure modes a generation run would surface. A reviewer who wants the stronger signal can regenerate any low-stakes skill from this branch.Propagation
Propagation to existing generated skills is still required and is not done here. Roughly 69
<slug>-dev-loopinstances exist, all predating this change, and per CLAUDE.md's "Promoting a rule into the template" the template only fixes drift forward. Every instance carrying a<!-- template-version: ... -->older than this commit needs a retrofit pass through the genericmaintain-dev-loopskill — a separate wave, not this PR. The three motivating issues in the instance repos are referenced above as motivation only and are deliberately not closed by this PR; they should be closed as their own repositories are retrofitted.drafted by Claude on behalf of Daniel Stephenson